Papers with layer-wise relevance propagation
Better Explain Transformers by Illuminating Important Information (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing explanations focus on the input and output of the Transformers, resulting in confusing results. |
| Approach: | They propose to highlight important information and eliminate irrelevant information by a refined information flow on top of the layer-wise relevance propagation method. |
| Outcome: | The proposed method outperforms baseline models on classification and question-answering datasets with over 3% to 33% improvement on explanation metrics. |
Explaining Text Similarity in Transformer Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Modern foundation models provide flexible text representations that enable the detection of semantic structure in vast amounts of unlabeled data. |
| Approach: | They propose to leverage layer-wise relevance propagation to understand the inner prediction mechanisms of NLP models by analyzing grammatical interactions, multilingual semantics, and biomedical text retrieval. |
| Outcome: | The proposed methods demonstrate their utility in three corpus-level use cases, analyzing grammatical interactions, multilingual semantics, and biomedical text retrieval. |
Refining Pretrained Word Embeddings Using Layer-wise Relevance Propagation (D18-1)
Copied to clipboard
| Challenge: | Recent research trend is to refine or fine-tune pretrained word embeddings. |
| Approach: | They propose a method for refining pretrained word embeddings using layer-wise relevance propagation using a neural network. |
| Outcome: | The proposed method achieves higher performance than the original vectors. |
Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to unsupervised style transfer lack fine-grained control of the influence from the target style. |
| Approach: | They propose a model that exploits the relevance of each output word to the target style . they pretrain a style classifier and train an attentional Seq2seq model to reconstruct input sentences . |
| Outcome: | The proposed model achieves state-of-the-art performance in terms of transfer accuracy and content preservation. |